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Method for describing the coaqposition of audio sig«ia 

«xe Invention relates to a method for deBcribing the 
s composition of audio signals, especially for 

the apatialization of MPBG-4 encoded audio signals in a 3D 
dotnaln* 

XO T.J^V T^"* """^"^ ^""-"^ ISO/IEC .144S6-3 and 
10 1449^-1 facilitates a wide variety of applications by sup- 
porting the representation of audio objects. For- the combi- 
natxon of the audio objects additional information - the so- 
called scene description ^ determines the placement in space 
and txme and is transmitted together with the coded audio 
15 object Bp 



20 



25 



30 



35 



A scene description is structured hierarchically and can be 
^presented as a graph, wherein leaf-nodes of the graph form 
the separate objects and the other nodes describes the proc 
esslng, e.g. positioning, scaling, effects. A node named 
"Sound- allows spatiali^ation of the audio signal in a 3D 
domain. A further node with the name -Sound2D- only allows 
spatialization on a 2D screen.. The use of the -Sound- node 
in a 2D graphical player is not specified due to different 
implementations of the properties in a 2D and 3D player. 

The. Sound2D node is defined as followed: 

Sound2D { 

«3»bsedPield SFPloat intensity 1 0 

eaiposedField SFVac2f location 0 0 

«q.osedPield spN^de source 

J SBBool apatialize TRUE 

and the Sound node is defined as followed: 
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Sotind { 





exposedPield 


SFVec3f 


direction 


0, 0, 1 




exposedField 


SFFloat 


intensity 


1.0 


5 


exposedField 


SFVec3£ 


location 


0. 0, 0 




exposedField 


SFFloat 


maxBack 


10.0 




exposedField 


SFFloat 


maxFront 


10.0 




exposedField 


SFFloat 


minBack 


1.0 




exposedField 


SFFloat 


minPront 


1.0 


10 


exposedField 


SFFloat 


priority 


0.0 




e^gposedPield 


SFNode 


source 


wmL 




field 


SFSool 


Epatialize 


TRUE 



} 



15 In the following the general term for all sound nodes 
(Sound2D, Sound and Direct iveSovmd) will be written in 
lower-case e.g. ^aound nodes . 

Please pay attention, to the fact that the Sound node is ac- 
20 tually a 3D node. 

In the sinplest case the Sound or Sound2D node is connected 
via an AudioSource node to the decoder output. The sound 
nodes contain the inteizazty and the location information. 
25 From the audio point of view a sound node is the final node 
before the loudspeaker mapping. In the case of several sound 
nodes, the output will be summed up. From the sys terns point 
of view the BO\and nodes can be seen as an entry point for 
the audio sub graph, A sound node can be grouped with non- 
30 audio nodes into a Transf oim node that will set its original 
location . 

With the pb&BeGroup field of the AudioSource node, it is 
possible to mark channels that contain itr^portant phase rela- 
35 tions, like in the case of ^^stereo pair", ^multichannel'^ 
etc. A mixed operation of phase related channels and non- 
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phase related chanixels is allowed. A spatiali^e field in the 
sound nodes specifies whether the sound shall be spatialized 
or not. This is only true for channels, which are not member 
of a phase group. 

The Sound2D ean spatialize the sound on the 2D screen. The 
standard said that the sound should be spatialized on scene 
of Size 2m X l.Sm in a distance of one meter. This explana- 
tion aeema to be ineffective because the value of the loca- 
tion field is not restricted and therefor^ the edund can 
also be positioned outside the screen size. We suppose to 
remove the confusing description. 



10 



X5 



20 



25 



30 



35 



The sound and DirectiveSound node can set the location eve- 
rywhere in the 3D space. The mapping to the existing loud- 
speaker placement can be done using single amplitude panning 
or more sophisticated techniques. 

Both Sound and Sound2D can handle multichannel inputs and 
basically have the same functionalities, but the Sound2D 
node cannot spatd.alizs a sound other than to the front. 

Prom games, cinema and TV applications we know, that it 
makes sense to provide the end user with a fully spatialized 
-SD-Sound- presentation, even if the video presentation is 
limited to a small flat screen in front. 

A first proposal is to add Sound and Sound2D to all scene 
graph profiles, i.e. add the Sound node to the SP2ra!rode 
group . 

But, one reason for not including the -3D- sound nodes into 
the 2D scene graph profiles is, that a typical 2D player is 
not capable to handle 3D vectors (SPVec3f type) , as it would 
be required for the Sound direction and location field. 
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Another reason is that the Sound node is specially'" designed 
for virtual reality scenes with moving listening points and 
attenuation attributes for far distance sound objects. For 
this the Listening point node and the Sound maxB&ck, max-- 
5 Front, ittlnBaGk and miaFront fields are defined. 

The proposal is to extend the old Sound2D node ox- to define 
a new Sound2Ddepth node. The Sound2Ddepth node should be 
simllsur the 6ound2D node but with an additional depth field. 

.0 

Sound2Ddepth { 



exposedPield 


SFFloat 


intensity 


1.0 


exposedField 


SFVec2f 


location 


0,0 


es^osedField 


SFFloat 


depth 


0.0 


ea^osedField 


SFNode 


source 


NUIiIi 


field 


SFBool 


spatlalize 


TKUE 



) 

The IntenBity field adjusts the loudness of the sotmd. Its 
20 value ranges from O.o to 1.0, and this value specifies a 
factor that is used during the playback of the sound 

The location field specifies the location of the sound in 
the 2D scene. 

2S 

The depth field specifies the depth of the sound in the 2D 
scene using the same coordinate system than the location 
field. The default value is 0.0 and it refers to the screen 
position. 

30 

The spatlalize field specifies whether the sound shall be 
spatialised. If this flag is set, the sound shall be spati- 
alized with the maucimum sophistication possible. 

35 The same rules for muJLtichannel audio spatialization apply 
to the Sound2Ddepth node as to the Souind (3D} node. 
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using the Souna2D node in a 2D scene allows presenting sur- 
rovmd sovmd, as the author recorded it. it is not possible 
to epatialise a sound other than to the front. Spatialize 
means moving the location of a tnonophonic signal due to user 
interactivities or scene updates. 



With the new proposal of a SouiidJ2Ddepth node it is possible 
to spatialize a sound also in the back, at the side or above." 
10 of the listener. Supposing the audio presentation system has 
the capability to present it. 

The invention allows the spatialization of the audio signal 
in a 3D domain, even if the player is restricted to 2D 
15 graphics. 



A furtlier embodiment ie as follows: 

20 The audio nodes -Sound', •Sound2D' and • directiveSound • .de- 
scribed in the MPB6-4 audio standard have besides , the func- 
tionality to present phase-related multichannel signals 
(^ichannel), the functionality to present accumulated mono- 
phonic sounds everywhere in the 2D or 3D space. Their posi- 
25 tion in these spaces can be addressed by using the -loca- 
tion '-field of these nodes. The location is a 2D- or 3D- 
vector. If the 'spatialize '-field of these nodes is set to 
'true', the sound will be spatialized depending on the -lo- 
cation' . Por the spatialization process different algorithms 
30 can be used, for example the amplitude panning. 

With these techniques it is only possible to position a 
sound as a point sound source. The sound has no 'width'.. 

Different methods are actually discussed to position a sound 
35 in a virtual reality world space, with a noticeable width. 
The approach of these algorithms is to describe a size or a 
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shape in a 3D coordinate system. 

There has been no approach for a user-oriented system. The 
following solution is to insert two new fields into the 
Sound and Sound2D nodes to control the width of the sound 

5 with an opening-angle relative to the listener. The angle 
has a vertical and a horizontal coit^jonent, 'widthHorizontal • 
and 'widthVertical' r ranging from 0 — 2« with the location 
as its center- The widthHorizontal is shown in Pig. 3- The 
widthVertical is 'similar to this with a 90 degree x-y- 

10 rotated relation. 

The width of a sound can be generated for example as de-. 
scribed in DE - A - 196 32 734 'methods and apparatus for 
generating a multichannel signal from a mono signal ' • 

15 

An application is for exairs»le the monophonic transmission of 
the violins in an orchestra instead its transmission as ste- 
reo signals r coupled to a fixed loudspeaker layout and no 
possibility to position it at a desired location. 

20 

According to a further embodiment a mapping of the coordi- 
nates is performed- An additional field dimensionMapping in 
the Sound2DDepth node defines a transf ormationr e.g, as a 
2 rows X 3 columns vector used to map the 2D context coordi- 
25 nate-system (ccs) from the ancestor's transform hierarchy to 
the origin of the node. 

The node's coordinate system (ncs) will be calculated as 
follows: 

nCB s CCS X dxmenaionMappingf , 

30 

The location of the node is a 3 dimensional position^ merged 
from the 2D input vector location and depth {location -x lo- 
cation. y depth} with regard to ncs- 

35 Example: The node's coordinate system context is {xi^ yi} . 
dimensionMapping is {l, o, 0, 0, o, l} . This leads to 
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a<=s=( xi, 0, What enables ti« » 

•^he y-dlmenslon to be napped to^„ °^ "^^^'='= 

depth. ^^^"^ "^^^ ««dlo movement m the 

field 'dimenslonMapping. :„ay be d«^- . ' 
same functionality could aleol^r MFPloat . The 

<*ata type -SFRotatlon. that Is J''^'^^ ""''^^ ^^^i** 

at an other mpbg-4 data type 
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Clalme 



1. Method for describing the composition of audio signals, 
which are encoded as separate audio objects, wherein 
the arrangement and the processing of the audio objects 
in a sound scene is described by nodes arranged hierar- 
chically in a scene description, wherein a node allows 
spatialization on a 2D screen using a 2D vector, 
charaoterised by describing a 3D position of an audio 
object using said 2D vector and a ID value describing 
the depth of said audio object. 

2. Method according to claim 1, characterized by 
controlling the width of an audio object fay using two 

15 additional fields inserted into the Sound and Sound2D 

nodes describiag an opening- angle relative to the lis* 
tener. 
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Abstract 

Method for describing the conqposition of audio signals, 
s which are encoded as separate audio objects. The arrangement 
and the processing of the audio objects in a sound scene is 
described by nodes arranged hierarchically in a scene de- 
scription. A node specified only for spatialization on a 2D 
screen using a 2D vector describes a 3D position of an audio 
10 object using said 2D vector and a ID value describing* the 
depth of said audio object . 

In a fxirther emboditnent two new fields are inserted into the* 
Sound and Sound2D nodes to control the width of the sound 
15 with an opening-angle relative to the listener. 
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